The invention relates to voice call quality testing.
Packet-based networks, in particular, Voice Over IP (VOIP) networks, are rapidly emerging as a viable alternative to traditional telephony (that is, circuit switched networks). VOIP is viewed as an attractive option for voice transport in that it allows live voice conversations to be integrated with existing IP data and image applications. To be a truly competitive alternative, VOIP must emulate the performance of traditional telephony and do so using a protocol that was optimized for data traffic. The characteristics of data traffic are quite different from those of voice traffic, however.
Unlike data traffic, voice traffic is extremely intolerant of delay and delay variation (or “jitter”), as well as packet loss. Much work has been done in the area of packet delivery to provide end-to-end Quality of Service (QoS). Service level agreements (SLAs) for VOIP, like those for conventional data IP networks, therefore tend to be based on conventional data network metrics, that is, guaranteed service levels are expressed solely in terms of packet level performance, e.g., packet loss, delay, jitter.
Another important aspect of voice communications quality that is not reflected in the data network metrics, however, relates to the sound of a voice call from the perspective of the listener. Standardized techniques exist for measuring this aspect of voice quality. Typically, to support voice communications, VOIP networks encode the audio and format the encoded audio into packets for transport using an IP protocol. Consequently, the results of these voice quality tests are greatly affected by choice of speech coding techniques.
One approach utilizes a standardized ranking system called the Mean Opinion Score (MOS). The MOS system uses a five-point scale: excellent-5; good-4; fair-3; poor-2; and bad-1. A level of quality indicated by a score of 4 is considered to be comparable to “toll quality”. A number of people listen to a selection of voice samples or participate in conversations, with the speech being coded by using the technique to be evaluated. They rank each of the samples or conversations according to the five-point scale and a mean score is calculated to give the MOS. Although the applicable ITU standard, ITU-T P.800, makes a number of recommendations regarding the selection of participants, the test environment, explanations to listeners, analysis of results, etc., because this type of voice quality testing is subjective, results tend to vary from test to test.
Algorithms for performing a more objective voice quality test have been developed as well. These objective techniques for testing voice transmissions in packet-based networks include an ITU standard based algorithm known as Perceptual Speech Quality Measurement (PSQM) and Perceptual Analysis Measurement System (PAMS), which was developed by British Telecom. Both of these perceptual test algorithms evaluate whether a particular voice transmission is distorted from the perspective of a human listener.
PSQM takes a “clean” voice sample and compares it to potentially distorted version, that is, a transmitted version, using a complex weighting that takes into account what is perceptually important to a human listener, for example, the physiology of the human ear and cognitive factors related to what human listeners are likely to notice. PSQM provides a relative score (on a scale of 1 to 15, with 1 corresponding to the highest score and 15 corresponding to the lowest score) that indicates how different the distorted signal is with respect to the reference from the perspective of the human listener.
PAMS is based on a perceptual model similar to that of PSQM and shares with PSQM the purpose of providing a repeatable, objective means for measuring voice quality, but uses a different signal processing model than PSQM and produces a different type of score as well. The scoring provided by PAMS includes a “listening quality” score and a “listening effort” score, both of which correlate to the MOS scores and are on the same scale of 1 to 5.